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(57) When video signals are composed so that one 
video signal can be defined by (i) inputting plural video 
signals, (ii) attaching a picture identifier to the respec- 
tive video signals for identifying the video signals, and 
(iii) composing some of the plural video signals, a pic- 
ture coding method including the following processes is 
proposed. 1. Prepare composing information such as a 
composing sequence. 2. Encode at least the video sig- 
nals having picture identifiers together with the compos- 
ing information. A coding apparatus using this method, 
a picture decoding method that decodes the video sig- 
nals coded with this method, a decoding apparatus 
using this decoding method are also proposed. A 
recording medium that records the data implementing 
the above methods is proposed. Thanks to the present 
invention, plural video signals can be coded in a high 
accuracy, and the composing information that is utilized 
for composing the plural video signals can be changed 
with < 
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Input a plurality of video signals 



Attach a picture identifier to the respective 
video signals for identifying them individually 
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Output composing Information for forming a composed 
video signal by composing a plurality of video signals 
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Encode at least the video signals to which picture 
identifiers are attached and composing information 
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Packetize and transmit it in a multiplex 
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Description 
Technical Field 

[0001 ] The present invention generally relates to cod- 
ing and decoding video signals, and more particularly to 
a method of coding plural pieces of picture information, 
an encoder using the method, a method of decoding 
and composing the coded plural digital picture data, an 
apparatus using this method, and a medium recording 
these methods. 

Prior Art 

[0002] Data compression (=encoding) is required for 
efficient storing and transmitting of a digital picture. 
[0003] Several methods of encoding are available as 
prior art such as "discrete cosine transform" (OCT) 
including JPEG and MPEG, and other wave-form 
encoding methods such as "sub-band", "wavelet", "frac- 
tal" and the like. Further, in order to remove redundant 
signals between pictures, a prediction method between 
pictures is employed, and then the differential signal is 
encoded by waveform encoding method. 
[0004] Recently, a method of coding and decoding 
has been employed so that a coding efficiency can be 
improved as well as each object constituting a picture 
can be independently coded, transmitted, and decoded 
at a reproduction side that composes the individual 
decoded object into a picture for displaying it. 
[0005] An advantage of this method is that objects can 
be arbitrary combined and composed thanks to an indi- 
vidual-object-coding method, whereby moving pictures 
can be re-edited with ease. This method also can save 
relatively unimportant objects from being reproduced 
depending on busyness of the transmission lines, 
capacity of the reproduction apparatus, or taste of users 
without negative influence to viewing the motion pic- 
tures. In other words, object-wise scalability is possible 
with this method. 

[0006] In order to embody the above function, the 
composing of individual objects should be changed with 
ease. In the prior art, information necessary for com- 
posing objects is included in the coded data of each 
object. The information about composing objects 
includes the depth of an object, placement thereof, and 
visibility thereof and the like. When re-editing is required 
or an object is required to be flexible such as being 
scaled down/up, the coded data must be analyzed first, 
and then the data relevant to the composing information 
is changed because the composing information is 
included in the coded data of the object. 

Disclosure of the Invention 

[0007] The present invention aims to provide a 
method of coding, a method of decoding and compos- 
ing, as well as an apparatus using the same methods. 



With these methods and apparatus, composing infor- 
mation can be changed with ease, when each object 
constituting a picture is independently coded and trans- 
mitted to a receiver which composes the plural pieces of 

5 picture information. More specifically, the present inven- 
tion aims to change the composing information with 
ease by saving the coded data from being analyzed. 
[0008] A picture coding method of the present inven- 
tion encodes video signals together with composing 

10 information through the following steps: 

(1) input plural video signals; 

(2) attach a picture identifier to the video signals 
individually for identifying each video signal; 

15 (3) when some of the plural video signals are to be 
composed into one single video signal, prepare rel- 
evant composing information such as composing 
sequence; and 

(4) encode the video signals to which at least pic- 
20 ture identifiers are attached and the composing 
information. 

[0009] The present invention can provide a picture 
encoding apparatus that employs this method. The 
25 apparatus comprises the following elements: 

(a) picture input means for inputting plural video 
signals; 

(b) picture identifier attaching means for attaching a 
30 picture identifier to the plural video signals individu- 
ally in order to identify each video signal; 

(c) composing information preparation means for 
outputting the composing information in order to 
compose the plural video signals; and 

35 (d) coding means for encoding the composing infor- 
mation and the video signals having at least the pic- 
ture identifiers. 

[0010] Another coding method of the present inven- 
40 tion encodes video signals and composing information 
through the following steps: 

(1) divide one picture into plural objects in order to 
form plural portions of the picture, whereby a single 

45 video signal is divided into plural video portion sig- 
nals; 

(2) attach a picture identifier to the video portion 
signals individually for identifying each video por- 
tion signal; 

so (3) prepare composing information including com- 
posing sequence so that the video portion signals 
are composed into one single video signal; and 
(4) encode the video portion signals having at least 
picture identifiers, and encode the composing infor- 

55 mation. 

[001 1] The present invention can also provide a pic- 
ture coding apparatus that employs the above method. 
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The apparatus comprises the following elements so that 
one single video signal can be formed with plural pic- 
tures by composing: 

(a) picture division means for dividing a video signal 
into plural video portion signals; 

(b) picture identifier attaching means for attaching a 
picture identifier to the video portion signals individ- 
ually for identifying each video portion signal; 

(c) composing information preparation means for 
outputting composing information in order to form a 
single video signal by composing the plural video 
portion signals; and 

(d) coding means for encoding at least the compos- 
ing information the video portion signals having the 
picture identifiers. 



information. 

[001 5] As described above, when elements, such as 
objects in a picture, defining the picture are coded and 
transmitted independently for composing each element 

5 into a picture, the composing information can be 
changed with ease and can be flexibly as well as arbi- 
trarily composed. Also when plural video portion signals 
are coded and transmitted for composing these plural 
signals into a new picture, the composing information 

w can be changed readily and flexibly as well as arbitrarily 
composed. As a result, interactive performance is 
noticeably improved. 



Brief Description of the Drawings 

15 

[0016] 



Fig. 1 is a flowchart depicting a method of coding 
plural video signals in a first exemplary embodi- 

20 ment of the present invention. 

Fig. 2 is a flowchart depicting a method of coding 
plural video signals in a second exemplary embod- 
iment of the present invention. 
Fig. 3 illustrates structures of coded picture data 

25 and the picture data to be decoded. Fig. 3(A) illus- 
trates first coded picture data. Fig. 3(B) shows mul- 
tiplexed data of packetized first and second coded 
picture data. Fig. 3(C) shows second coded picture 
data. 

30 Fig. 4 depicts a content example of the composing 
information used in the exemplary embodiment of 
the present invention. 

Fig. 5 is a flowchart depicting a method of decoding 
the multiplexed data of plural video signals utilized 

35 in a third exemplary embodiment. 

Fig. 6 is a block diagram illustrating a decoding and 
composing apparatus utilized in the third exemplary 
embodiment of the present invention. 
Fig. 7 is a schematic chart illustrating an example of 

40 composing plural video signals, where each object 
is composed. 

Fig. 8 is a timing chart of composing and displaying 
a picture, which illustrates the decoding operation 
in the exemplary embodiment of the present inven- 
45 tion. 

Fig. 9 is a block diagram illustrating a decoding and 
composing apparatus utilized in a fourth exemplary 
embodiment of the present invention. 



[0012] A decoding and composing method that can 
decode and compose the coded video signal under- 
gone the above process comprises the following steps: 

(V) receive plural coded data; 

(2*) decode the plural coded data; 

(3') extract the picture identifiers attached to the 

coded data. 

On the other hand, the composing information is 
received, and plural coded data are decoded based on 
this information and the picture identifiers, then finally 
one decoded picture is obtained. 
[0013] The present invention provides a picture 
decoding and composing apparatus that employs the 
above method and comprises the following elements. 

(a*) receiving means for receiving plural coded data; 
(b') decoding means for decoding the coded data 
received by the receiving means; 
(C) picture identifier extraction means for extracting 
the picture identifiers attached to the coded data 
that are to be decoded by the decoding means; 
(d') composing information receiving means for 
receiving the composing information that is utilized 
to compose the plural coded data; and, 
(e') composing means for composing the plural 
coded data based on at least the picture identifiers 
and the composing information. 

[0014] The composing information includes at least a 
composing sequence and picture identifiers of the 
object pictures to be composed. According to the com- 
posing sequence, a depth and location designated by 
the picture identifier of a portion picture are determined, 
whereby plural reproduced portion pictures are com- 
posed. The picture decoding and composing apparatus 
further comprises input means, through which only the 
composing information independent of the coded data 
is modified, and then the reproduced portion pictures 
are composed according to the modified composing 



so Detailed Description of Exemplary Embodiments 

(Exemplary Embodiment 1) 

[001 7] The first exemplary embodiment is described 
55 with reference to Figs. 1. 3 and 4. Fig. 1 is a flowchart 
depicting a method of coding plural video signals in a 
first exemplary embodiment of the present invention. 
Fig. 3 illustrates a bit-stream data-structure of the result- 
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ant digitally coded plural video signals. Fig. 4 shows a 
description example of the data. 
[0018] In Fig. 1, input a plurality of video signals on 
Step 1 1 . The plural video signals will be detailed later 
with reference to Figs. 3(A) and 3(C). On Step 12, 
attach a picture identifier to each input video signal to be 
identified. In Fig. 3(A), an identifier (ID) 302 is attached 
to picture data 304 having time (i). In Fig. 3(C). an iden- 
tifier (ID) 323 is attached to picture data 328 having time 
('*)■ 

[0019) On Step 13 in Fig. 1, output the composing 
information for forming a composed video signal by 
composing plural video signals. Fig. 3(B) illustrates mul- 
tiplexed data of packetized coded picture data shown in 
Figs. 3(A) and 3(C). The composing information packet 
319 is the one supplied on Step 13. 
[0020] On Step 14, encode the video signals at least 
having the picture identifier, and the composing infor- 
mation. The resultant coded video signals are 304 and 
330 to which identifiers (ID) 302 and 307 as well as time 
identifiers (TR) 303 and 329 are respectively attached. 
Another resultant coded video signal is 328 that has 
identifiers (ID) 323. 327 and time identifier (TR) 324. 
[0021 ] On Step 1 5 in Fig. 1 , packetize and then trans- 
mit in multiplex the coded video signals shown in Figs. 
3(A) and 3(C). The packetized form is shown in Fig. 
3(B). In this case, a packet 319 is independently pro- 
vided for containing the composing information, while 
video packets 311 and 315 are provided for the video 
signals. A composition parameter is described in this 
packet 319 and transmitted, the parameter tells how to 
compose and output the video signals shown in Figs. 
3(A) and 3(C). 

[0022] Fig. 4 shows an example of the above case, 
and composing information 401 is depicted. A composi- 
tion time 402 records the time when the video signals 
are composed. A composition order (composing 
sequence) 403 includes picture identifiers (OID), time 
identifiers (TR), depth (CO) and placement (XPOS, 
YPOS). Composing information for blending filter is rep- 
resented with 404, a frame memory management is 
represented with 405. 

[0023] In this case, a composing sequence is listed as 
part of composing information. When plural pictures to 
be composed differ in size, a coordinates converting 
parameter can be listed so that compositions rich with 
variety by changing the composing information can be 
achieved with ease. 

(Exemplary Embodiment 2) 

[0024] The second exemplary embodiment is 
described with reference to Figs. 2, 3 and 4. Fig. 2 is a 
flowchart depicting a method of coding plural video sig- 
nals in a second exemplary embodiment of the present 
invention. There are several different points from Fig. 1. 
First, a work is added, i.e. input a picture, and divide it 
into plural portion pictures. Fig. 3 shows a bit-stream 



data structure resulting from coding digitally the plural 
portion pictures. This is the same as described in the 
first exemplary embodiment. Fig. 4 shows also the 
same operation steps as the first exemplary embodi- 

s ment, and provides an example of data description. 
[0025] On Step 21 in Fig. 2, input video signals to be 
coded, and on Step 22 divide the video signals into plu- 
ral portion pictures with a method that divides a picture 
into elements such as objects and backgrounds within 

10 the picture. 

[0026] Regarding Step 23 to Step 26, the same oper- 
ation as in Fig. 1 except substituting the portion video 
signals for the video signal. One video signal is divided 
into plural portion video signals and composing informa- 

15 tion, and the coded picture data thereof are supplied in 
the form of bit-stream having the structure shown in Fig. 
3(B). 

[0027] Through these steps, one picture can be accu- 
rately transmitted, and also the picture can be edited 
20 with ease by re-writing the composing information at 
decoding operation. 

(Exemplary Embodiment 3) 

25 [0028] Fig. 5 is a flowchart depicting decoding and 
composing in the third exemplary embodiment. Fig. 6 
illustrates a decoding and composing apparatus 
employing the method shown in Fig. 5. Fig. 7 illustrates 
schematically how the two pictures are composed. An 

30 operation of the apparatus shown in Fig. 6 is described 
with reference to Figs. 5 and 7. 
[0029] Fig. 6 lists the following elements: input termi- 
nal 100, separator 101, control decoder 102. video 
decoders 103 and 104, frame memory bank 105, com- 

35 poser 106, and display unit 107. 

[0030] An operation of the picture decoder and com- 
poser constructed as above is described hereinafter. 

(1) Input received picture data and composing infor- 
40 mation to the input terminal 100 (Step 51 in Fig. 5.) 

(2) Send the composing information to the control 
decoder 102 using the separator 101 via a line 108, 
and convert the information into a control signal 
(Steps 52, 55 and 56.) 

45 (3) Input the coded picture data to the video decod- 
ers 103 and 104 (Step 53.) Two video decoders are 
prepared in this exemplary embodiment in order to 
make the description simple; however, arbitrary 
numbers of video decoders can be provided. 

so (4) Decode the coded picture data in the video 
decoders 103 and 104, and reproduce different pic- 
tures, then store them in the frame memory bank 
105 (Steps 53 and 54.) The video decoders utilized 
in this exemplary embodiment employ "inter-frame 

55 motion compensation" and "discrete cosine trans- 
formation" that are typical functions of MPEG 1 ; 
however, the video decoder embodying the present 
invention is not limited to this type. 
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(5) Based on a control signal sent from the control 
decoder 102 via a line 1 1 7. access the object repro- 
duced pictures for composing in the frame memory 
bank 1 05, and send the subject pictures to the com- 
poser 106 via a line 118. 

(6) In the composer 106. compose the reproduced 
pictures sent via a line 1 15 following the control sig- 
nal sent via the line 118 (Step 57.) 

(7) Output the composed picture on the display 107 
when necessary (Step 58.) 

[0031] The above operation is described with refer- 
ence to the specific pictures in Fig. 7. A first reproduced 
picture 201 has been reproduced by the video decoder 
1 03. A second reproduced picture 202 has been repro- 
duced by the video decoder 104. 
[0032] The first reproduced picture 201 and the sec- 
ond reproduced picture 202 are composed in the com- 
poser 106 into a picture 203. The control signal sent via 
the line 118in Fig. 6 assigns the first picture 201 afront 
view, and the second picture 202 a background. 
[0033] The operation is further detailed with reference 
to Fig. 3 used in the first and second exemplary embod- 
iments. Af irst coded picture data shown in Fig. 3(A) and 
a second coded picture data in Fig. 3(C) are composed. 
[0034] The picture data 304 having time (i) is headed 
by a sync, signal 301 followed by the identifier (ID) 302, 
time identifier (TR) 303, and coding data in this 
sequence. The identifier (ID) and the time identifier (TR) 
are indicated in 5 bits and 16 bits respectively. 
[0035] The picture data 330 having time (i+1) is also 
headed by a sync, signal 306 followed by the identifier 
(ID) 307, the time identifier (TR) 329, and coded data. 
Because Fig. 3(A) contains coded data in a same pic- 
ture family, all the identifiers (ID) are the same; however, 
because the time identifiers (TR) depend on display 
times, they take different values. 
[0036] The data structure of the second coded picture 
shown in Fig. 3(C) is the same that in Fig. 3(A); how- 
ever, the identifiers used in Fig. 3(C) take different val- 
ues from those in Fig. 3(A) to distinguish themselves 
from each other. 

[0037] The data shown in Fig. 3(B) are produced 
through packetizing and multiplexing the picture data. 
The first coded picture data shown in Fig. 3(A) and the 
second coded picture data in Fig. 3(C) are separated 
into plural packets before being multiplexed. For 
instance, the picture 304 having time (i) is separated 
into video packets 31 1 and 315, the picture 328 having 
time (i) is separated into video packets 313 and 317, 
and then, they are multiplexed. Headers 310, 312, 314 
and 316 are attached to respective packets. These 
headers include a packet identifier that is written in 6 
bits fixed-length code. Different packet identifiers are 
attached in order to distinguish the first coded picture 
data from the second coded picture data. The compos- 
ing information is also packetized into a composing 
information packet 319, and then multiplexed. The 



packet 319 also has a header to which a packet identi- 
fier different from that of video packets is attached. In a 
packet identifier list (OAT) 309 lists the packet identifiers 
of the composing information packets and the packet 
5 identifiers of the packets containing the subject pictures 
to be composed. 

[0038] The multiplexed data shown in Fig. 3(B) is to be 
received (Step 51 in Fig. 5.) By referring to the packet 
identifier list, the separator (demultiplexer) 101 in Fig. 6 

10 sorts out video packets and composing information 
packets, where each packet has a packet identifier. 
Then restore the selected video packets to coded pic- 
ture data, and input them into the video decoders 103 
and 104 (Step 52.) 

15 [0039] On the other hand, the composing information 
packet is restored to the composing information and 
sent to the control decoder 102, where the composing 
information is analyzed and converted into control infor- 
mation (Steps 55, 56.) The control decoder in this exem- 

20 plary embodiment is a script interpreter, which analyzes 
the script of composing information as follows: 
[0040] Fig. 4 depicts the composing information 401 , 
where composition time 402, composing sequence 403, 
information for blending filter 404 and frame memory 

25 control information 405 are listed. The composing infor- 
mation defined as above is utilized as follows for com- 
posing a picture. 

[0041] The composition time 402 indicates the time 
when a picture is composed with hour, minute and sec- 

30 ond. When the time comes in accordance with the timer 
of the apparatus, the picture is composed and displayed 
according to the composing sequence 403 and the 
information for blending filter 404. The composing 
sequence 403 includes picture identifiers (OID), time 

35 identifiers (TR), depth (CO), placement (XPOS, YPOS). 
When the composition time comes, the picture desig- 
nated by the picture identifier and time identifier is 
accessed in the frame memory bank 1 05 to be read out, 
and the picture is composed following the depth and 

40 placement of the relevant picture. 

[0042] In this exemplary embodiment, "CO=0" indi- 
cates the deepest background of the picture, "CO=r 
indicates a front picture next to the background "0", and 
"CO=2" indicates further front picture to the layers of 

45 "CO=1" and "CO=0". The placement indicates the pic- 
ture position on the coordinates in the displaying area. 
In this exemplary embodiment, "(0, 0)" indicates a 
center of the displaying area. 
[0043] The information for blending filter 404 includes 

so a coefficient of a filter that filters boundaries of the com- 
posed picture. In this exemplary embodiment, the coef- 
ficient of filter is 3x3. The frame memory control 
information 405 controls the opening and securing of 
the frame memory bank 105 in Fig. 6. In this exemplary 

55 embodiment, a picture having a picture identifier (0) 
releases the occupied memory. 
[0044] The timing with which a picture is composed 
and displayed is described hereinafter with reference to 



5 



9 

the composing information in Fig. 4. Fig. 8 illustrates 
part of the timing. The X axis indicates a display time of 
the composed picture. The composing information is 
indicated from 513 to 517. A picture having a picture 
identifier 2 (<OID>{2)) is indicated from 501 to 505. A 5 
picture having an identifier 3 ((OID){3)) is indicated in 
506 and 507. A picture having an identifier 4 ( < OID ) {4}) 
is indicated from 508 to 512. 

[0045] At the time 15", the pictures 501 , 506, 508 and 
the composing information 513 are available, and 10 
according to the information 513, a picture is composed 
and displayed. When the picture 508 cannot be 
received or reproduced due to the business of the trans- 
mission line or limited capability of the apparatus, only 
the pictures 501 and 506 are composed. At the time 15 
"t8", the pictures 504, 51 1 and the composing informa- 
tion 516 are available, but the picture having the picture 
identifier 3 is not available. The information 516 in this 
case is depicted in Fig. 4, and the time identifier of the 
picture having the identifier 3 is indicated "5" (i.e. 20 
<OID){3}, (TR>{5) of the composing sequence 403 in 
Fig. 4.) In other words, at the time "t8", rt is enough to 
compose the pictures 504, 506 and 51 1 . When a picture 
of which time identifier is designated is unavailable, a 
picture having a time identifier prior to this one can be 25 
used for composing. For instance, if the picture 51 1 hav- 
ing the identifier 4 and time identifier 8 is not available, 
the picture 510 replaces the picture 51 1 and is used for 
composing. At the time "t6" and 17", the same process 
is necessary. 30 
[0046] The time chart in Fig. 8 represents the case 
where composing information is continually received. 
When the composing information is not continually 
received, the composing information most recently 
received is utilized for composing until the information is 35 
updated. For example, the composing information 514, 
515 and 516 are not received, the information 513 is 
kept using for composing pictures until the information 
51 7 is received. In this case, the time identifier attached 
to the information 51 3 is replaced with those attached to 40 
each reproduced picture when composing is practiced. 
[0047] The information for blending filter 404 and the 
frame memory control information 405 are not always 
included in the composing information 401. The picture 
placement information and time identifier are also not 45 
always included in the composing sequence 403. When 
the time identifier is not available, pictures to be com- 
posed can be identified with the time identifier attached 
to each reproduced picture. The composing information 
can include data other than the above. so 
[0048] A decoding process and a composing appara- 
tus are described with regard to pictures, the same 
process and apparatus can be applicable when audio 
signals are composed. 

55 

(Exemplary Embodiment 4) 

[0049] Fig. 9 is a block diagram illustrating a decoding 



10 

and composing apparatus utilized in a fourth exemplary 
embodiment of the present invention. The structure and 
the basic operation are the same as those in Fig. 6. An 
input terminal 601 is added to the structure shown in 
Fig. 6. A remote controller or a keyboard can be used as 
the input terminal 601. Part of the composing informa- 
tion is changed via the input terminal 601 . For instance, 
data of depth out of the composing sequence 403 is 
changed so that the background can replace the front 
view. 

Industrial Applicability 

[0050] When elements, such as objects in a picture, 
defining the picture are coded and transmitted inde- 
pendently for composing each element into a picture, 
and also when plural video signals are coded and trans- 
mitted for composing these signals into a new picture, 
the present invention provides the composing informa- 
tion that tells how to compose the plural video signals. 
The composing information is transmitted together with 
the video signals so that the information can be 
changed with ease at decoding, and the new picture 
can be flexibly and arbitrarily composed. As a result, a 
noticeable advantage can be obtained such that interac- 
tive performance satisfying both sender and receiver 
can be practiced in editing. 

List of reference marks in the drawings 
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control decoder 
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Claims 





1. A method of coding a picture comprising the steps 

of: 

(a) inputting a plurality of video signals; 

(b) attaching picture identifiers to each video 
signal for identifying thereof; 

(c) outputting composing information that is uti- 
lized for defining one video signal by compos- 
ing said plurality of video signals; and 
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(d) encoding the composing information and 
the video signal to which at least the identifier is 
attached. 

2. A picture coding apparatus comprising: 5 

(a) input means for inputting a plurality of video 
signals; 

(b) picture identifier attaching means for attach- 
ing a picture identifier to said video signal for w 
identifying the video signals; 

(c) composing information preparation means 
for outputting composing information that is 
used for composing said plurality of video sig- 
nals; and is 

(d) encoding means for encoding the compos- 
ing information and said video signal to which 
at least the picture identifier is attached. 

3. A method of coding a picture wherein a video signal 20 
is composed with a plurality of pictures, said 
method comprising the steps of: 

(a) dividing a video signal into a plurality of por- 
tion video signals; 25 

(b) attaching a picture identifier to each said 
portion video signal for identifying said portion 
video signal; 

(c) outputting composing information that is uti- 
lized for defining the video signal by composing 30 
said plurality of portion video signals; and 

(d) encoding said portion video signals to 
which at least the picture identifier is attached, 
and the composing information. 

35 

4. A picture coding apparatus that composes a video 
signal with a plurality of pictures, said apparatus 
comprising: 

(a) dividing means for dividing a video signal 40 
into a plurality of portion video signals; 

(b) picture identifier attaching means for attach- 
ing a picture identifier to said portion video sig- 
nals for identifying said portion video signal; 

(c) composing information preparation means 45 
for outputting composing information to define 
the video signal by composing said plurality of 
portion video signals; and 

(d) encoding means for encoding the compos- 
ing information and said portion video signals so 
to which at least the picture identifier is 
attached. 

5. A method of decoding and composing a picture 
comprising the steps of: 55 

(a) receiving a plurality of coded data; 

(b) decoding said coded data; 



19 A1 12 

(c) extracting a picture identifier attached to the 
coded data; 

(d) receiving composing information that is uti- 
lized for composing the plurality of coded data; 
and 

(e) composing the plurality of coded data 
based on at least the picture identifiers and the 
composing information. 

6. A picture decoding and composing apparatus com- 
prising: 

(a) receiving means for receiving a plurality of 
coded data; 

(b) decoding means for decoding the received 
plurality of coded data; 

(c) picture identifier extracting means for 
extracting a picture identifier attached to the 
coded data to be decoded; 

(d) composing information receiving means for 
receiving composing information that is utilized 
for composing the plurality of coded data; and 

(e) composing means for composing the plural- 
ity of coded data based on at least the picture 
identifier and the composing information. 

7. A picture decoding and composing apparatus com- 
prising: 

(a) receiving means for receiving coded data of 
a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing and display means for compos- 
ing and displaying the reproduced pictures 
according to the composing information. 

8. A picture decoding and composing apparatus com- 
prising: 

(a) receiving means for receiving coded data of 
a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
duced pictures according to the composing 
information, 

wherein the composing information includes at 
least the picture identifier of the picture to be 
composed and a composing sequence, 
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wherein the composing sequence determines 
a depth of a picture designated by the picture 
identifier, 

whereby the plurality of and different pictures 
are composed. 

9. The picture decoding and composing apparatus of 
Claim 8, 

wherein the composing information includes at 
least the picture identifier of the picture to be com- 
posed and a composing sequence, 
wherein the composing sequence determines a 
depth and a placement of the picture designated by 
the picture identifier, 

whereby the plurality of and different pictures are 
composed and displayed. 

10. A picture decoding and composing apparatus com- 
prising: 

(a) receiving means for receiving coded data of 
a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
duced pictures according to the composing 
information, 

wherein the composing information includes at 
least the picture identifier of the picture to be 
composed, the time identifier and a composing 
sequence, 

wherein the composing sequence determines 
a depth of the picture designated by the picture 
identifier and the time identifier, 
whereby the plurality of and different pictures 
are composed and displayed. 

11. The picture decoding and composing apparatus of 
Claim 10, 

wherein the composing information includes at 
least the picture identifier of the picture to be com- 
posed, the time identifier and a composing 
sequence, 

wherein the composing sequence determines a 
depth and a placement of the picture designated by 
the picture identifier and the time identifier, 
whereby the plurality of and different pictures are 
composed and displayed. 

12. The picture decoding and composing apparatus of 
Claim 8, 9, 10 or 1 1, wherein a reproduced picture 
corresponding to the picture identifier included in 
the composing information is unavailable, and 
wherein said apparatus is free from composing the 



reproduced picture. 

13. The picture decoding and composing apparatus of 
Claim 10 or 1 1 , wherein a reproduced picture corre- 

5 spending to the picture identifier included in the 
composing information is unavailable, and wherein 
said apparatus utilizes a picture for composing, said 
picture having the same picture identifier that of the 
reproduced picture and being reproduced most 

w recently. 

14. A picture decoding and composing apparatus com- 
prising: 

15 (a) receiving means for receiving coded data of 

a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
utilized for composing the pictures, 
20 (b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
duced pictures according to the composing 

25 information, 

wherein the composing information includes at 
least a composition time, 
wherein a reproduced picture is composed fol- 
lowing the composing information when a timer 
30 of said apparatus comes to the composition 

time. 

1 5. A picture decoding and composing apparatus com- 
prising: 

35 

(a) receiving means for receiving coded data of 
a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 

40 utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
45 duced pictures according to the composing 

information, 

wherein said apparatus composes the plurality 
of and different pictures following composing 
information most recently received before the 
so composing information is renewed. 

1 6. A picture decoding and composing apparatus com- 
prising: 

55 (a) receiving means for receiving coded data of 

a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
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utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
duced pictures according to the composing 
information, 

wherein the composing information includes at 
least information for blending filter, 
wherein said apparatus filters a composed pic- 
ture according to the information for blending 
filter when said apparatus composes the repro- 
duced pictures following the composing infor- 
mation. 

1 7. A picture decoding and composing apparatus com- 
prising: 

(a) receiving means for receiving coded data of 
a plurality of and different pictures to which a 
picture identifier and a time identifier are 
attached, and composing information that is 
utilized for composing the pictures, 

(b) decoding means for decoding the pictures, 

(c) producing means for producing a plurality of 
and different reproduced pictures; and 

(d) composing means for composing the repro- 
duced pictures according to the composing 
information, 

wherein said apparatus further comprises input 
means through which the composing informa- 
tion is changed, said apparatus composes the 
reproduced pictures following the changed 
composing information. 

18. A picture transmitting method comprising the steps 
of: 

(a) packetizing coded data of a plurality of and 
different pictures having a picture identifier and 
a time identifier, and composing information 
that is utilized for composing the plurality of and 
different pictures; 

(b) attaching a packet identifier to a packet of 
the plurality of and different pictures, and to a 
packet of the composing information; 

(c) multiplexing the respective packets having 
the packet identifiers; and 

(d) transmitting the respective packets together 
with an identifier list containing the packet iden- 
tifiers. 

1 9. A picture decoding and composing apparatus com- 
prising: 

(a) a separator; 

(b) a picture decoder; 

(c) a composing information decoder; 



(d) a frame memory; and 

(e) a composer; 

wherein said apparatus (i) packetizes coded 
data of a plurality of and different pictures to 

5 which a picture identifier as well as a time iden- 

tifier are attached, and composing information 
that is utilized to compose the plurality an differ- 
ent pictures, (ii) attaches packet identifiers dif- 
ferent with each other to the respective 

w packets, and multiplexes the respective pack- 

ets, (iii) inputs transmitted data together with 
packet related information including the identifi- 
ers, (iv) selects the packets having the packet 
identifiers included in the packet related infor- 

15 mation using said separator, (v) transmits 

packet data of the plurality of and different pic- 
tures to said picture decoder, as well as trans- 
mits packet data of the composing information 
to said composing information decoder, (vi) 

20 decodes the coded data of the plurality of and 

different pictures into a reproduced picture 
using said picture decoder, and stores the 
resultant decoded data into said frame mem- 
ory, (vii) converts the composing information 

25 into a control signal using said composing 

information decoder, (viii) transmits the repro- 
duced picture stored in said frame memory to 
said composer following the control signal, and 
(ix) composes the plurality of and different pic- 

30 tures following the control signal using said 

composer. 

20. The picture decoding and composing apparatus of 
Claim 19, wherein the composing information fur- 
35 ther includes information of releasing and securing 
said frame memory, wherein said apparatus 
releases and secures said frame memory accord- 
ing to the composing information. 

40 21 . A picture coding and decoding system comprising: 

(a) picture data transmission means for 



45 



50 



55 



(i) packetizing coded data of a plurality of 
and different pictures to which at least a 
picture identifier is attached, and compos- 
ing information that is utilized to compose 
the plurality of and different pictures; 

(ii) attaching packet identifiers different 
with each other to the respective packets, 
and multiplexes the respective packets; 
and 

(iii) transmitting the respective packets 
together with a packet identifier list con- 
taining the packet identifiers. 

(b) a separator for 
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(i) selecting the packets having the packet 
identifier included in the packet identifier 
list; 

(ii) separating packet data of the plurality of 
and different pictures and packet data of s 
the composing information, and outputting 
respective packet data; 

(c) a picture decoder for decoding coded data 

of the plurality of and different pictures tapped 10 
off from said separator into a reproduced pic- 
ture, 

(d) a frame memory for storing an output from 
said picture decoder, 

(e) a composing information decoder for con- 15 
verting the composing information tapped off 
from said separator into a control signal, and 
outputting the control signal, and 

(f) a composer for composing the plurality of 
and different pictures using the reproduced pic- 20 
ture stored in said frame memory following the 
control signal. 

22. A recording medium for recording data with which 
at least one of methods of Claim 1 , 3, 5 and 18 is 25 
executed. 
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FIG. 2 



input a video signal 
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FIG. 5 
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Compose the plurality of coded data based on at least 
the picture identifiers and the composing information 
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Display the resultant composed data when necessary 



15 



EP0 949 819 A1 




16 



EP 0 949 819 A1 




17 



EP0 949 819 A1 



CO 

(3 

LL 



CO 



8n 



o 



CM 



Ov 



CO 



00 



to 



^0 



CD 
5) 



"a 



to 



CO 



A 

g 
o 



g 
o 

V 



A 

9 
o 

V 



.2 

«♦—» 

CD 

o> 

8. 



A 
V 



18 



EP0 949 819 A1 




19 



EP0 949 819 A1 



INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/JP97/01975 



A. CLASSIFICATION OF SUBJECT MATTER 
Int. CI 6 H04N7/24 

According to International Patent Classification (IPC) or to both national classification and IPC 



B. FIELDS SEARCHED 



Minimum documcntition searched (classification system followed by classification symbols) 

Int. CI 6 H04N7/24-H04N7/68 



Documentation searched other thao minimum documentation to the extent that such documents are included in die fields searched 

Jitsuyo Shinan Koho I960 - 1997 

Kokai Jitsuyo Shinan Koho 1975 - 1997 



Electronic dats Use consulted during the international search (name of data base and, where practicable, search terms used) 



C. DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to datm No. 



1-19, 21-22 



Minoru Eido "Trend in the movement MPEG4 of 
standardization of moving picture encoding {in 
Japanese)" Technical Research Report of IEICE r 
Vol. 95, No. 469, 
January 18, 1996 (18. 01. 96), 
(Tokyo) p. 55-60 

Eiji Kasuya, Hisashi Miyamori, Hideyoshi 
Tominaga "Coding of image content and its 
application to edit processing function (in 
Japanese)" Technical Research Report of 
Information Processing Soc . of Japan, Vol. 96, 
No. 17, 

February 16, 1996 (16. 02. 96), 
(Tokyo) p. 29-36 

JP, 6-98313, A (Sony Corp.), 

April 8, 1994 (08 . 04. 94) , 

Column 1, lines 38 to 41 & US, 5534928, A 

& EP, 588586, A2 



1-19, 21-22 



13 



|X | Further documents are listed in the continuation of Box C | [ See patent family annex. 



• Special categories of died documents: 

"A" document defining the general state of the art which is not considered 

to be of particular relevance 
"E" earlier document but published oa or after the international filing date 
"L" document which may throw doubts oo priority daimfs) or which is 

cited to establish the publication date of another citation or other 

special reason (as specified) 

"O" document referring to an oral disclosure, use, exhibition or other 
means 

"P" document published prior to the international filing date but later than 
the priority date claimed 



T later document published after the international filing date or priority 
date and not in conflict with the application but cited to understand 
the principle or theory underlying the inventioo 

"X" document of particular relevance; the claimed invention cannot be 
considered novel or cannot be considered to involve an inventive 
step wees the document is taken alone 

**Y*° document of parties tar relevance; the claimed invention cannot be 
considered to involve an inventive step when the document is 
combined with one or more other such documents, such combination 
being obvious to a person skilled in the art 

document member of the same patent family 



Date of the actual completion of the international search 
August 29, 1997 (29. 08. 97) 



Date of mailing of the international search report 

September 9, 1997 (09. 09. 97) 



Name and mailing address of the ISA/ 

Japanese Patent Office 

Facsimile No. 



Authorized officer 



Telephone No. 



Form PCT/1SA/210 (second sheet) (July 1992) 



20 



EP0 949 819 A1 



INTERNATIONAL SEARCH REPORT 



International application No. 

PCT/JP97/01975 



C (Continuation). DOCUMENTS CONSIDERED TO BE RELEVANT 



Category* 



Citation of document, with indication, where appropriate, of the relevant passages 



Relevant to claim No. 



Hiroshi Yasuda "MPEG /International standard of 
Multimedia encoding (in Japanese)" Maruzen 
(Tokyo) (1994) p. 148-171, particularly refer to 
p. 153, 165, 166 

JP, 8-79742 , A . (Nippon Telegraph & Telephone 
Corp. ) , 

March 22, 1996 (22. 03. 96), 
Column 2, lines 32 to 36 (Family: none) 

Minoru Eido "Trend in standardization of MPEG4 
(in Japanese)" The Journal of the Inst, of Image 

Electronics Engineers of Japan, Vol. 25, No. 3, 

June 25, 1996 (25. 06. 96) , 
(Tokyo) p. 223-228, particularly refer to 

p. 225, right column, lines 13 to 15 

JP, 4-10884, A (Mitsubishi Electric Corp.), 
January 16, 1992 (16. 01. 92) (Family: none) 



14, 18-19, 
21-22 



15 



16 



16 



Form PCT7ISA/210 (continuation of second sheet) (July 1992) 



21 



